Exploring the Network Structure of Virtual Communities: A Web Structure Analysis of a Korean-American Community

We introduce new general-purpose agent software, which crawls the web using userspecified “on-the-fly” social theory-based criteria to identify and analyze web sites in a particular virtual community. It then renders information about the sites into data in standard formats suitable for additional analysis by statistical or network software. In this study, a simple “two-layer” centrality measure is used to identify 250 highly interconnected websites forming a virtual community on Korean-American affairs. We then examine how theoretical propositions about centrality and prestige in the literature on social networks apply to cyberspace. Our analysis suggests that web sites with greater centrality within the virtual community are more prominent and popular, attracting a greater number of visitors. Despite the recent explosion of interest in the internet among sociologists and the prominent role that the analysis of the internet plays in advancing diverse sub-disciplines in sociology, particularly social network analysis, there are fewer empirical, especially quantitative, sociological analyses of the internet than one might hope or expect (DiMaggio et al., 2001; Wellman and Haythornthwaite, 2002). Perhaps the greatest barrier to the analysis of the internet may be the limited access to suitable data from the internet. For instance, a search on the topics of “web” or “internet" and “social networks” in the EBSCO academic database allows us to locate only a handful of quantitative studies of virtual communities in sociology journals, each using their own ad hoc methodology of collecting data. Furthermore, most quantitative analysis of the internet tend to focus on email correspondence, chatrooms, message boards, or newsgroups as their research settings, using the content analysis methods developed for traditional text documents (Krippendorff, 2004). In contrast, quantitative analysis of the World Wide Web (WWW) is nearly nonexistent apparently because of considerable methodological challenges researchers face. This would seem paradoxical, since by its nature, content on the web is widely accessible and already in electronic form. Moreover, the “social" aspects of the web are obvious in the sense that web sites contain not only text content but also link to other web sites with related content (Stewart, 2003). The most persuasive explanation for the shortfall in quantitative analysis of the WWW is the lack of suitable technologies, which allow one to crawl the web using sociologically relevant criteria for sampling and to code the downloaded content into data suitable for statistical analysis (Weare and Lin, 2000). This paper presents a new technology for addressing those challenges, based upon agent software developed by one of the authors. The software is intended to download web sites in a virtual web community, while compiling quantitative information about the sites.